Step 1: Sign up / get assigned EC2 instance
Step 2: SSH into EC2 instance
Step 3: Start up the cluster by running the following commands
In [ ]:
cd ~/nexus/esip-workshop/docker/infrastructure
docker-compose up -d cassandra1
docker logs -f cassandra1
Step 4: Wait to see "INFO 01:57:02 Starting listening for CQL clients on /0.0.0.0:9042…" and then run
In [ ]:
docker-compose up -d
Step 5: Verify services are running using
In [ ]:
docker ps
There should be 9 containers running: 3 solr, 3 zookeeper, and 3 cassandra. If they are not all running, run docker-compose up -d again.
Step 6: Start the analysis cluster
In [ ]:
cd ~/nexus/esip-workshop/docker/analysis
docker-compose up -d
Step 7: Verify services are running using
In [ ]:
docker ps
There should be an additional 5 containers running: 3 mesos-agent, 1 mesos-master, 1 nexus-webapp
Step 8: Start the ingestion cluster
In [ ]:
cd ~/nexus/esip-workshop/docker/ingest
docker-compose up -d
Step 9: Verify services are running using
In [ ]:
docker ps
There should be an additional 9 containers running: 3 kafka, 3 xd-container, 1 redis, 1 mysqldb, 1 xd-admin
Ingest 2017 AVHRR data that has been staged under /home/ndeploy/ingest/data/avhrr/2017
Step 10: Deploy the stream to Spring XD using nx-deploy-stream.sh available inside xd-admin container
In [ ]:
docker exec -it xd-admin /usr/local/nx-deploy-stream.sh --datasetName AVHRR_OI_L4_GHRSST_NCEI --dataDirectory /usr/local/data/nexus/avhrr/2017 --variableName analysed_sst --tilesDesired 1296